Development and Delphi validation of instrument for the preparation of a GMP audit of a cosmetic contract manufacturer in the UAE

Outsourcing is rapidly increasing in the cosmetic industry. When cosmetic manufacturing activities are outsourced, product quality control and assurance procedures are important to guarantee the safety of the final products. One method to ensure the compliance of potential contract manufacturers with Good Manufacturing Practices (GMP) is auditing. Auditing can be confirmed by using GMP questionnaires. The current study aims to develop an effective instrument, in the form of a questionnaire, and to validate its content using the Delphi method. A modified Delphi method used an expert panel to develop a questionnaire-based audit preparation instrument. In the Delphi questionnaire round, 50 experts from cosmeceutical industries, relevant authorities, and universities assessed the content validity of the audit preparation instrument. The Delphi questionnaire round targeted 70 experts from the cosmetics industry; 50 completed the questionnaire, giving a response rate of 71.4%. Their agreement level on the quality of the instrument items ranged between 56 and 96%. Of the 52 items, 47 (90.4%) met the predefined criterion for an agreement rate of at least 75%. The proposed audit preparation instrument demonstrated good content validity, and the expert panel participating in the Delphi questionnaire round made a few minor suggestions for modifications. The modified Delphi method used for the content validation of the instrument proved to be suitable. However, based on the panel’s feedback, additional research is needed to ensure the maximum applicability and practicality of the instrument.


Methods
Study design. The current study aimed to develop and validate an effective instrument that can assist in the preparation of the cosmetic contract manufacturer auditing process in the UAE. This study employed a modified Delphi method. This method enables group judgment on a subject that lacks accurate and/or complete information drawn from research 17,18 . A Delphi survey involves a set of expert panelists completing and evaluating a questionnaire in rounds, thereby successively highlighting those items for which there is high agreement, i.e., consensus. After the responses have been summarized after each round, those items on which the panelists were unable to agree are removed from all subsequent questionnaire rounds. This process enables the continuous refinement of the consensus among experts on the given topic. As participant anonymity is ensured throughout the procedure, this method enables controlled feedback to be obtained 19 . This method is often used to develop consensus on guidelines or standards for which there is little or no research-based evidence 20 .
Recruitment of participants. The following two participant types were included in this study: Scientific committee. Initially, a scientific committee was set up. This consisted of seven experts, with three of these coming from the relevant regulatory authorities while the remaining four were specialists attached to Ajman and Al Ain universities. To be chosen for the scientific committee, the members needed to have a PhD as well as a strong record of experience and knowledge in the following fields: the cosmetics industry, pharmaceutical technology and good manufacturing practice auditing (GMP). Furthermore, we included 10 experts from the case company, a medium-sized cosmetics producer that uses several contract manufacturers for its finished products. The aim hereby was to firstly, ensure that the scientific committee had sufficient expertise and experience on the investigated topic and secondly, prevent the findings from being unrepresentative of the larger population. Next, the scientific committee conducted a review of the questionnaire's first version developed by the researchers. The members were hereby asked to recommend changes to the questionnaire before it was sent to the panel of experts. They were also requested to evaluate the criteria that were used to choose which experts should take part in the Delphi rounds. They could also recommend additional members, as long as these fulfilled the selection criteria. The professionals suggested by the research team were then sent invitations asking them to join the scientific committee. Upon agreeing to take part, each committee member received the relevant documents by email for them to review.
Expert panel. The selection of experts can be done based on very few guidelines or rules 21 . Generally, an expert group can be assembled from individuals who have relevant knowledge or expertise, e.g. academics or specialists, or professionals who are active in the field 22 .
To be chosen for the Delphi panel, potential respondents had to be experts, either international or national, and possess the right skills and knowledge in the auditing of good manufacturing practices in the cosmetics and personal care product sector, demonstrated by a proven record of scientific papers published in peer-reviewed academic journals or policy reports; participation at cosmeceutical science conferences was also considered as relevant. Furthermore, the respondents needed to be proficient English speakers and able to complete the questionnaire in English without the need for translation. In particular, each panel expert needed to be able to do perform the following: • Clarify the principles, processes and techniques of GMP and their importance for the cosmetics and personal care products industry. • Perform third-party audits in the cosmetics and personal care products industry as per ISO 22716 GMP. Sampling and sample size. As there are few guidelines indicating how many experts are ideally required to be on a Delphi panel, the researchers settled on at least 20 experts 23,24 . Consequently, 70 experts were contacted and asked if they would agree to participate in the Delphi consensus of whom 50 expressed interest. Various professionals with deep knowledge of ISO 22716 GMP auditing and certification were chosen for the expert panel using purposive sampling. They were then registered on the researchers' network and sent an email inviting them to take part. If they agreed, the link to the online questionnaire was sent to the respondents via email. The cover page of the questionnaire provided information on the study's purpose as well as the risks and benefits to the participants. By completing the questionnaire, the participants were assumed to have given their informed consent.

Development of the audit preparation instrument.
To begin with, the extant literature on the topic was reviewed by first performing a search of full-text publications on the Scopus and PubMed databases. The topic was evidence of best practices in ISO 22716 GMP auditing. Once studies had been identified, relevant information was extracted on the best practices for optimal cosmetic products production, control, storage and shipment. An Excel spreadsheet was used to collate all relevant statements. The items for the first Delphi questionnaire version were developed based on the literature review results. These were then sent to the scientific committee for their opinion, after which a brainstorming session was held to address any issues. The final draft consensus document comprised an instrument for evaluating the contract manufacturers of sterile as well as non-sterile finished cosmetics products. www.nature.com/scientificreports/ Delphi questionnaire round. This study employed a one modified round Delphi survey; this is considered suitable if there is a considerable body of primary literature on the topic being investigated 25 . The draft version of the questionnaire comprised a list of statements sent via email to the 50 panel members. In the document, the study objectives were clearly described, and specific instructions on participation were included. The draft of the audit preparation instrument was sent to the panel experts in February 2021, and they were asked to evaluate it by ranking each item's significance on a 5-point Likert scale (from 1 = strongly disagree to 5 = strongly agree). The panel experts also gave their opinions on whether any items were missing from the instrument and whether they could suggest any further modifications or had any other feedback on the questionnaire's content. The respondents could clarify their responses using the text fields located at the end of each section. The Delphi round aimed to also highlight any redundancies or identify issues that influenced the comprehension of each individual statement. A research assistant calculated each item's response frequencies and anonymously recorded them in a database. The following consensus criterion was used: if more than 75% of the panel ranked an item as either (agree) or (strongly agree), it was considered essential and retained in the instrument. If this criterion was not met, the item was considered non-essential and excluded. There have been similar expert consensus definitions used throughout the literature [26][27][28] .
Data collection. The Google Online Survey tool was employed to operate the online questionnaire's Delphi round. An email, containing an introductory letter as well as the link to the online questionnaire, was sent to the chosen experts. The experts were first asked to answer a question on whether or not they were willing to participate in the questionnaire, and only those who indicated their willingness were asked to fill in the questionnaire. Two reminders, sent two weeks apart, were sent to any non-responsive experts. The data were collected between February 2021 and April 2021.
Statistical analysis. The data were analysed using SPSS version 26. The sample's demographics and baseline characteristics were summarized using frequencies and percentages. Mean ± Standard Deviation (± SD) was used to summarize the quantitative variables (GMP items). Differences in agreement level between groups were evaluated using Mann-Whitney and Kruskal-Wallis tests. A p-value of less than 0.05 was selected to demonstrate statistical significance.

Ethics approval and consent to participate. This study received approval from Ajman University's
Institutional Ethical Review Committee (P-H-S-2021-2-9). All methods were carried out in accordance with relevant guidelines and regulations. The study aim was clearly presented on the questionnaire cover page, and all respondents were informed that their participation was completely voluntary. If the Participants proceeded to the second page of the questionnaire, were considered to have given their written informed consent. The participants' identities were not recorded and the confidentiality of their data was guaranteed.

Results
Baseline characteristics of the participants. Table 1 shows the demographic and baseline characteristics of the different types of experts who participated in the study. A total of 50 professional experts participated in the panel. Of these, 10% (n = 5) had less than 1 year of work experience, 50% (n = 25) had 1 to 5 years work experience, 16% (n = 8) had 6-10 years work experience, 8% (n = 4) had 11-15 years work experience and 16% (n = 8) had ≥ 16 years. Among the participants, 20 (40%) had bachelor's or equivalent degrees, 27 (54%) had master's degrees and 3 (6%) held PhDs. Slightly more than half (54%) worked in public (government) organization, 36% in private organizations and 10% worked in both public and private organizations. The roles of the experts were as follows: 6 (12%) R&D Product Specialists, 6 (12%) Quality Control Supervisors, 10 (20%) Quality Assurance Specialists, 13 (26%) Production Technologist and ISO Representatives, 4 (8%) ISO 22,716 Auditors, 3 (6%) Data Entry Encoders, and 8 (16%) Cosmetic Safety Assessors. Of the participants, 70% (n = 35) were from the UAE and 30% (n = 15) from the EU. Phase 1: developing the audit preparation instrument. To pinpoint evidence of best practices in ISO 22716 GMP auditing, a literature search was conducted, whereby 22 relevant publications were found. Following a title and abstract screening, 14 of the 22 were chosen and the scientific committee submitted a further 4 . As a result, the study was able to identify 18 publications on the production, control, storage and shipment of cosmetic products.
The Guidelines on Good Manufacturing Practices (ISO 22716: 2007) were initially used to create the framework. These were specifically formulated to ensure that the cosmetics industry implements good manufacturing practices. Hence, they outline cosmetic products' pharmaceutical supply chains and are applicable to all supply chain members, from the raw material producers to the manufacturers of cosmetics products. Specifically, they explicate the special requirements to be considered in constructing a quality management system (QMS) in accordance with the principles of GMP and ISO 22716.
Aligning with the ISO 22716: 2007 taxonomy, the brainstorming session with the scientific committee determined ten dimensions for GMP in the cosmetics manufacturing industry (Table 2). These encompassed production, raw materials, premises, equipment, personnel, laboratory control, labelling and packaging, customer complaints, recording, and others. Subsequently, the scientific committee convened to come to an agreement on these categories and describe each one according to their expertise. The brainstorming session used the draft audit preparation instrument, which mainly measured the items using closed-ended questions, i.e. the responses were in the yes/no format. This was chosen after consultation with professionals from the case company, and it www.nature.com/scientificreports/ has been established that this method is convenient for respondents to give their opinions. Using several openended questions in the questionnaire could have burdened the respondents due to the time needed to complete it. Following the brainstorming session an effort was made to identify and merge or remove redundant statements as well as statements that incorporated similar constructs. Hereby, of the initial 121 statements, 84 were either removed or grouped, eventually producing 15 statements with consensus for incorporation into the draft audit preparation instrument. Overall, 52 statements were taken to be used in the draft audit preparation instrument. Building on the Assessment of pre-existing guidelines on Good Manufacturing Practices (ISO 22716: 2007) as well as the articles gathered during the literature search, the final draft audit preparation instrument comprised 52 items categorized in the abovementioned ten dimensions. During the Delphi round voting process, these 52 statements were sent to the experts on the panel (Fig. 1).
Phase 2: Delphi questionnaire round. Following the voting for Delphi round, the experts agreed upon 47 of the 52 statements (i.e. > 75% of experts denoting "agree" or "strongly agree"). The final guideline document thus incorporated these 47 statements. The panel also reached consensus to omit five items from the final document. The mean score distributions for the individual dimensions of the questionnaire are given in Table 3. Table 4 presents the distribution of the agreement rate in association with the demographic details and baseline characteristics of the respondents. There were no statistically significant differences in the agreement rates originating from the demographic information or baseline characteristics of the respondents.

Discussion
The establishment of the ISO 22716 GMP guidelines was a significant milestone in the development of a worldwide standard for the safety of cosmetics and personal care products. In particular, the standard focuses on technical, human, and administrative factors that can influence the quality of cosmetic products. Against this backdrop, the current study aimed to work towards a national consensus on cosmetics and personal care product safety by developing an instrument to prepare for cosmetic contract manufacturer auditing. Similar GMP questionnaires have been shown to be a useful aid in preparing for audits 14 , and numerous cosmetics manufacturers already employ such instruments. Nonetheless, thus far no study has investigated the use, contents, and validity of these GMP questionnaires.
The findings of the current study demonstrate that the developed audit preparation instrument has reasonably good content validity. Nearly all the quality items (47 of 52; 90%) were rated as essential by all the experts on the panel; the remaining 5 items did not meet the pre-defined criterion of at least 75% agreement. Based on these findings, 5 items were judged not relevant to the instrument and were excluded. While the Delphi method conventionally suggests a new Delphi questionnaire round at this point to conduct an additional assessment of agreement among the experts 19 , this study stipulated the criterion that if the agreement level was found to be below 75%, the particular item would be removed from the instrument. Hence, no further Delphi questionnaire rounds were performed.  • Evaluation of each quality item on a fivepoint Likert scale (i.e. >75% of experts denoting "agree" or "strongly agree"). www.nature.com/scientificreports/ The first draft audit in the brainstorming session received some criticism from the members of the scientific committee. Specifically, seven of the experts said there was too much detail in some of the questions, while others felt that the yes/no format provided insufficient information. Hence, these experts recommended using open-ended questions. Based on this feedback, text fields located at the end of each section was provided in the final audit instrument where the experts could clarify their responses in the Delphi round. However, in the early stages of the study, the case company decided that convenience in completing the audit preparation instrument was paramount, and thus yes/no questions were mostly used in its development.
A thorough knowledge base is necessary for the development of a GMP questionnaire 29 . However, the expert panel consulted in the Delphi round was relatively small (n = 50). Despite its small size, each panel member had experience in both GMP and the cosmetic industry, which gave them an in-depth perspective on the context of the research. Further, the inclusion of experts from the authorities and universities ensured the diversity of the panel. Finally, the Delphi questionnaire round received a very good response rate of 71.4%. In prior research, Delphi rounds involved expert panels ranging from 4 to 3000 individuals 17 , while several studies using the Delphi method involved relatively small expert panels 18,21 .
Building and Facilities section of the audit instrument ensures that the locations where products are manufactured and stored do not interfere with the quality of cosmetic products. The building and facilities section provides plant design and construction principles. It sets sanitary conditions under which manufacturers need www.nature.com/scientificreports/ to develop and store cosmetic products 30 . Through these principles, consumers can enjoy products that have undergone proper storage, adequate cleaning, and protection from toxic surfaces and environments. At the same time, the section has principles guiding the environment in which workers operate. Further, they emphasize ample water supply, proper sewage disposal, toilet facilities, hand-washing amenities, and rubbish clearance. Notably, the section ensures that consumers use cosmetic products manufactured under hygienic conditions. The equipment section of audit instrument contains principles that guide the maintenance of utensils used in manufacturing, storing, and distributing cosmetic products. Precisely, the principles guide how manufacturers and distributors need to handle any type of equipment in relation to the contamination of cosmetic products. It emphasizes that cosmetic products need to stay clean to avoid contaminating products during manufacturing, storage, and distribution 30 . Therefore, consumers use products that are free from toxins and impurities originating from equipment used.
The personnel section of audit instrument involves every individual who handles cosmetic products at any level of production or distribution. This section stipulates that cosmetic manufacturers need to implement guidelines and programs that ensure disease control, cleanliness among all workers, proper education and training, and adequate supervision 30 . That way, consumers access products whose manufacture involves hygiene, expertise, and keen observation. Therefore, the personnel section of GMP is essential in ensuring those involved in the manufacture, storage, and distribution respect the health and safety of consumers.
Raw Materials section of audit instrument have numerous provisions. First, raw materials need to undergo inspection and be handled in a manner that ensures cleanliness 31 . The measure enhances a clean start during the manufacturing process. Second, raw materials need not contain microorganisms that cause disease in humans 31 . This guideline helps to protect consumers from using harmful cosmetic products. Third, materials with ingredients that are susceptible to contamination, such as aflatoxin, need to comply with the Drug Administration policies. This guideline protects consumers from using products with high levels of toxic chemicals. The final guideline is that raw materials need to stay in storage facilities that protect them from contamination 31 . The guideline ensures the use of clean raw materials for manufacturing. All the policies ensure that the raw materials become end-products that are safe for consumers' bodies.
Production section has provisions that guide the conditions under which companies manufacture cosmetic products. The section emphasizes the appropriate time, temperature, pH, and acidification 32 . The section guidelines ensure that each product's production process follows the recommended conditions. Consumers enjoy products whose production entailed the use of appropriate conditions. Laboratory controls section involve testing the chemicals involved in the production of cosmetic products to check the levels of each chemical in the ingredients. With that in mind, laboratory control guidelines demand that components undergo investigation, sampling, retesting if needed, and thorough results analysis 33 . The guidelines ensure that consumers use products that are scientifically proven to be safe for human use.
Documentation section ensures traceability of all development, manufacturing, testing, and distribution activities 34 . The guidelines in this section ensure that whenever consumers need to ask about the manufacture of a product, the company will conveniently retrieve the records indicating its journey from development to sale.
The packaging and labeling section ensures that end-products are protected from contamination and mixing 31 . The main aim of the section is to protect the consumer from using cosmetic products that have toxins or are mixed up due to poor packaging and wrong labeling.
Complaints section demands that companies need to record and review each complaint 35 . The complaints will prompt the companies to recall or re-strategize the production of the involved cosmetic products. The section gives a voice to consumers since it ensures that manufacturers meet their quality demands.
Audit preparation instruments, such as the one developed here, have been shown to offer considerable support during the contract manufacturer auditing process. The current research will continue by investigating how well a completed audit preparation instrument reflects the reality during an audit. To achieve this, the answers provided by the company will be compared to the actual conditions observed while auditing. This comparison will offer valuable information on the applicability and practicality of the preparation instrument that can be integrated into future research. Furthermore, gathering feedback from contract manufacturers on the process of completing the instrument will also give crucial information on its usefulness.
The study results offer guidance for cosmetic companies aiming to incorporate cosmetics GMP when outsourcing their production to contract manufacturers, thereby ensuring consistency. Cosmetic companies operating outside the UAE may also find these results useful for implementing cosmetics GMP, although these findings primarily focus on the UAE context. However, elaboration and validation of an instrument such as described in this study is only a fraction of what it takes to obtaining a reliable and adequate instrument to be applied in its respective field-further studies are necessary to assess the psychometric properties of the instrument.

Conclusion
The developed audit preparation instrument showed good content validity, and the expert panel participating in the Delphi questionnaire round only had a few minor suggestions for modifications. The content validation of the preparation instrument was done via the modified Delphi method, which proved to be suitable. However, there was some criticism of the instrument, and additional research is needed to ensure maximum applicability and practicality. This initial study involved systematically developing an audit preparation instrument, and it achieved good results; this research direction is to be continued in future studies with the aim of producing a system for validated quality management evaluation. www.nature.com/scientificreports/

Data availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.